Medical viewing system and image processing for integrated visualisation of medical data

ABSTRACT

A medical viewing system comprising data acquisition means for acquiring image data in an image of an object surface and processing means for integrating clinical data with the image data, comprising processing means for processing the image data, whereby to identify a reference surface approximating the object surface and reference points on said reference surface; constructing a map, called distance map, comprising one or several distance transformed surface(s), from the reference surface, formed of image points that correspond univocally to reference points of the reference surface; estimating, at the location of the image points of the map, clinical data, and combining the clinical data and the image data at the location of the reference points, so that to integrate the clinical data in the image data; said medical viewing system further comprising image visualisation means for visualising the object images and/or the processed images.

FIELD OF THE INVENTION

The present invention relates to a medical viewing system and to animage processing method for integrated visualisation of medical imagedata relating to an anatomical element. The invention further relates toa medical examination apparatus having such a medical viewing system andto a computer program product having instructions for carrying out themethod steps. The invention finds its application in the field ofmedical imaging and, more especially, in the field of x-ray medicalimaging.

BACKGROUND OF THE INVENTION

A primary aim of medical imaging is to present medical image data in aform that is useful for the clinician. Initially this aim was fulfilledby providing the clinician with accurate representations of ananatomical feature of interest. There are many techniques now availablefor producing three-dimensional (3-D) medical image data representinganatomical features of interest to the medical practitioner. Variousmethods of processing and representing that medical image data have alsodeveloped. Increasingly, visualisation apparatus is interactive,allowing the clinician to control the view that is presented. Almost alltechniques currently used to render and visualise 3-D medical image datadepend on slicing or projecting data using conventional rectangularcoordinates (x, y, z) of the image. The images may further be“re-sliced” in any oblique plane going through the volume. Otherapproaches make use of a “curved multi-planar reformatting” in which thex-axis is replaced by any curvilinear path seen in a planarcross-section of the image, while the other dimensions of the volume areunchanged. Other systems allow the user to extract active surface modelsclosely fitting the boundaries of an organ as acquired in a 3-D medicalimage. As processing and visualisation techniques have become moresophisticated, it has become desirable to represent not only theanatomical feature of interest itself but, in addition, other associatedclinical data. This associated clinical data could be:

-   -   additional clinical data associated with the surface of        interest: for example, it could be useful to provide an image        not only of the surface of the skull, but also of the thickness        of bone at various points, or    -   additional anatomical image data relating to organs, vessels,        etc. which are associated with the surface of interest: for        example, on a representation of the heart it could be useful to        represent, in addition, the coronary arteries.

An anatomical feature could be visualised together with a representationin numerical form of associated clinical data. However, the medicalpractitioner can more easily interpret the represented information ifthe clinical data are integrated into the visual representation that ismade of the anatomical feature of interest. In the case of associatedanatomical image data, clearly it is desirable to represent theseadditional data in a manner integrated with the representation of theanatomical feature of interest.

A publication entitled “Integrated Visualization of QuantitativeInformation with Anatomical Surfaces”, pp. 195-200, in “ComputerAssisted Radiology, Proceedings of the International Symposium onComputer and Communication Systems for Image Guided Diagnosis andTherapy”, CAR'95, Berlin, Jun. 21-24, 1995, Springer, Karel, byZuiderveld et al., proposes an approach for integrating visualizationsof anatomical surfaces with quantitative data. According to the proposalof Zuiderveld et al, at numerous points over the anatomical surface ofinterest, the maximum, minimum or mean value of a given clinical ismeasured over a certain distance along the normal to the anatomicalsurface at that point. For each surface point, the clinical data ofinterest, mean, minimum, maximum, is evaluated by considering samples,for instance voxels, that are evenly spaced along the normal to thesurface at that point, and that are within a certain distance from thesurface. The calculation can take into account samples outside thesurface of interest, which are said to be along the surface normal,and/or samples inside the surface of interest, which are said to bealong the reverse surface normal. The measured clinical data are codedand integrated into the representation of the anatomical surface ofinterest as a texture on the displayed image, in this case by use ofcolour.

Unfortunately, when the technique proposed in the above-citedpublication is applied, so as to integrate clinical data into arepresentation of a curved anatomical surface, the method is prone toproduce an integrated representation, which is misleading, ambiguous, orimpossible to interpret. It is particularly the case where theanatomical surface of interest has a generally spherical shape, andwhere a clinical data of interest is measured along different reversenormals to the surface, in order to be displayed at the points ofintersection of said normals with the spherical surface. It has beenfound that first clinical data values measured on a first normal, fordisplay at a respective first intersection point of the surface, may beinfluenced by second clinical data values measured on a second normal,when said second clinical data values are measured at particularlocations on said second normal with respect to said first normal.

Incidentally, unless the contrary appears from the context, in thepresent document: the expressions “anatomical feature” and “anatomicalsurface” are intended to be read broadly so as to designate any featureor surface in the body, whether human or animal, whether a vessel, anorgan, a part of a vessel or organ, or anything else, and includeartificial elements implanted into or attached to the body; theexpressions “clinical parameter data” and “clinical data” both designatedata representing the value of one or more parameters of clinicalinterest, for example, rate of blood flow, thickness of surface,temperature, local blood perfusion, etc.; the expression “anatomicalimage data” and “image data” both designate image data representing thewhole or a part of an anatomical feature; and the expression “surfacenormal” includes the reverse surface normal.

SUMMARY OF THE INVENTION

The present invention has for an object to provide a medical viewingsystem having means for visualizing an anatomical surface of interest inan integrated fashion with associated clinical data, while avoidingvarious unwanted artefacts. In particular, the present invention has foran object to provide means of processing medical image data so as toenable improved integrated visualisation of a curved anatomical surfaceof interest and clinical data associated with that surface, and to avoidthe problems inherent in the approach by Zuiderveld et al.

The technical features of such a medical viewing system are recited inclaim 1.

The medical viewing system can be implemented as a specially programmedgeneral-purpose computer. The medical viewing system can be aworkstation. The present invention further provides an image processingmethod, which has steps to be performed by the processing means of themedical viewing system. This method comprises steps of processingmedical image data for visualizing an anatomical surface of interest inan integrated fashion with associated clinical data, without unwantedartefacts. The present invention yet further provides a computer programproduct having a set of instructions, when in use on a general-purposecomputer, to cause the computer to perform the steps of theabove-described method. The present invention still further provides amedical examination apparatus incorporating medical imaging apparatus,data processing system putting into practice the above-described methodto process medical image data obtained by the imaging apparatus, andmeans for visualising the image data produced by the method. Thevisualisation means typically consists of a monitor connected to thedata processing apparatus. Advantageously, the workstation and medicalimaging system of the present invention are interactive, allowing theuser to influence clinical data that are evaluated and/or the manner inwhich evaluated data is to be visualised.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention and additional features, which may be optionally used toimplement the invention to advantage, are described hereafter withreference to the schematic figures, where:

FIG. 1 is a diagram of a curved surface of interest and normals at twopoints of said surface;

FIG. 2 is a diagram illustrating basic components of an embodiment ofmedical viewing system, incorporated in a medical examination apparatus;

FIG. 3A and FIG. 3B are diagrams illustrating the construction ofdistance transform surfaces from the reference surface; and FIG. 3Cillustrates the problem of magnification that is solved by theinvention;

FIG. 4A is a flow diagram showing the main steps of a medical image dataprocessing method according to a preferred embodiment of the invention;and FIG. 4B is a flow diagram illustrating in detail the steps 2 to 5 ofFIG. 4A.

DESCRIPTION OF EMBODIMENTS

The invention relates to a medical viewing system for the visualizationof an anatomical surface of interest in an integrated fashion withassociated clinical data. The present invention will be described indetail below with reference to embodiments applied to an integratedvisualisation of curved surfaces of an organ together with other medicalfeatures or with clinical data. In the following detailed description, apreferred embodiment of the present invention will be described in whichthe anatomical feature of interest is the heart and it is the whole or apart of the surface of the epicardium (heart muscle) which is theprincipal anatomical surface to be visualised. However, the presentinvention can be applied to other curved anatomical_surfaces, such asthe following curved surfaces: the inner surface of the right ventricle,the outside surface of a vessel, inside surface of the colon, etc. In acase where the anatomical surface to be visualised is the epicardium, itcan be desirable to produce an integrated visualisation of this surfacetogether with the coronary arteries, or together with clinical parameterdata, e.g. rate of blood flow, relating to those arteries. The outsidesurface of the heart muscle can be extracted using known techniques,even in a coarse fashion, and a representation thereof generated, andclinical data relating to the coronary arteries can then be projectedonto the coarse representation. The integrated representation providesuseful data to the medical practitioner in a form that can beinterpreted in an easy manner.

Although medical imaging technology is well developed, currenttechniques are inadequate when applied to the “visualisation of curvedsurfaces together with clinical data”. The problem can be betterunderstood from consideration of FIG. 1 that represents a curvedanatomical surface to be processed in an integrated fashion withassociated clinical data. This anatomical surface of interest, RS, showsa generally spherical shape, giving a circular cross-section. It isassume that a clinical data of interest is measured along the reversesurface normals N_(A) and N_(B) in order to be displayed at two points Aand B on the surface, as taught by the Zuiderveld et al. approach. Ifthe Zuiderveld et al. approach is used, then the calculation for bothpoints A, B can be affected by the value at point O, at the centre ofthe circle, where the surface normals cross. Thus, the value taken bythe clinical data in question, at a given point, influences the finalrepresentation at two different locations, rendering the representationambiguous. The problem is particularly acute in a case where it is themaximum or minimum of the clinical data that is being measured, and inthe case where said maximum or minimum value occurs at point O.Moreover, if the clinical data of interest is evaluated inwards alongthe reverse surface normals N_(A) and N_(B) up to a distance thatexceeds the radius of the circle, then the value at data point P cancontribute to the surface representation at point B and the value atdata point Q can contribute to the surface representation at point A. Insuch a case, the relative order of the points P, Q has been reversedwhen they are mapped onto the surface of interest. Thus, when using theZuiderveld et al. approach, the resulting integrated visualisation ofthe surface together with the clinical data will be misleading.

The medical viewing system and an image processing method of the presentinvention permits to avoid the artefacts produced by the Zuiderveld etal. approach. A preferred embodiment of the present invention will nowbe described with reference to FIGS. 2 to 4.

FIG. 2 shows the basic components of an embodiment of an image viewingsystem in accordance to the present invention, incorporated in a medicalexamination apparatus. As indicated schematically in FIG. 2, the medicalexamination apparatus typically includes a bed 10 on which the patientlies or another element for localising the patient relative to theimaging apparatus. The medical imaging apparatus may be a CT scanner 20.The image data produced by the CT scanner 20 is fed to data processingmeans 30, such as a general-purpose computer. The data processing means30 is typically associated with a visualisation device, such as amonitor 40, and an input device 50, such as a keyboard, pointing device,etc. operative by the user so that he can interact with the system. Theelements 10-50 constitute a medical examination apparatus according tothe invention. The elements 30-50 constitute a medical viewing systemaccording to the invention. The data processing device 30 is programmedto implement a method of analysing medical image data according topreferred embodiments of the invention.

FIG. 4A is a flow diagram showing the steps in the preferred method ofprocessing medical image data in order to enabling improved integratedvisualisation of a curved anatomical surface and associated clinicaldata.

The image data input to the method is, in this example, 3-D computedtomography image data obtained for a subject heart is the image datainput to the method. The medical image data consists of a large numberof data relating to points (voxels), each corresponding to a respectiveposition within the patient's body. The preferred method furthercomprises steps:

S0 for preprocessing the image data. In step S0, the input image datamay be subjected to conventional pre-processing, for example, toeliminate noise.

S1 for calculating a segmented object surface. In step S1, the outersurface of the heart muscle is identified from within the image data viaa segmentation process as illustrated by the segmented curved surface RSin FIG. 3A to 3C. In the segmentation process, a 3-D surface is defined,which models the outer surface of the heart muscle. This 3-D segmentedsurface may be a surface defined by linking together points in themedical image data, which have the same intensity value, typically thesame grey level, hence called iso-surface. This permits of segmentingthe object with respect to a background that has a different grey level,or with respect to another organ. Alternately, this segmented surfacemay be obtained by linking together points that answer to a segmentationcriterion. In another technique, the 3-D surface may be obtained as anactive model providing a best fit to the heart muscle, or otheranatomical object under consideration. Yet further, this 3-D surface canbe user-defined, typically by operation of the pointing device or otheruser input device 50 shown in FIG. 2. Techniques for modelling a surfaceby an iso-surface are described, for example, in the “Handbook ofMedical Imaging, Processing and Analysis”, edited by Isaac N. Bankman,Academic Press, chapter 5 “Overview and Fundamentals of Medical ImageSegmentation” by Jadwiga Rogowska Techniques for producing an activemodel of an anatomical object are also well-known, for example by thedescription in the publication entitled “General Object ReconstructionBased on Simplex meshes” by Herve Delingette, in the InternationalJournal of Computer Vision, 32, 111-142, 1999.

S2 for calculating a reference surface. In a step S2, the segmentedobject surface is processed to yield a 3-D simplified surface, whichapproximates the segmented object surface. Preferably, the segmented 3-Dsurface is smoothed, using known techniques, to remove corners or highlycurved portions. The smoothed segmented surface is called “ReferenceSurface” and is denoted by RS hereafter.

Said simplified surface may be submitted, but not necessarily, to anoperation of discretisation. In an embodiment, this operation permits ofobtaining a 3-D surface closely approximated by a polyhedron referred toas “reference polyhedron”, wherein the 3-D simplified surface isdecomposed into small elements, called “patches” or “facets”, which arenot necessarily plane. In other embodiments, the reference surface RScan even be a mere approximation of the organ shape such as a sphere oran ellipsoid for the heart, a cylinder for the colon, etc.

If the reference polyhedron is used as reference surface, and showsplane facets, the normals to those facets are calculated. If thereference polyhedron is used as reference surface, and shows patches,the normals to those patches are approximated by an average normal. Ifthe reference surface RS shows neither patches nor facets, the normalsto a number of, or to all voxels, are estimated. This estimation isperformed by calculating the tangent surface at each considered voxeland then by calculating the normal to this tangent surface. Each facetor each patch in the reference polyhedron, approximating the 3Dsegmented surface, can be characterised by the (x,y,z) Cartesiancoordinates of its centroid, by the components (u,v,w) of the outwardnormal vector to the facet or patch, and by a set of adjacentneighbouring centroids. In other embodiments, each voxel of thesimplified reference surface RS is also characterised by its (x,y,z)Cartesian coordinates, by the components (u,v,w) of an outwardapproximated normal vector at this point, and by a set of adjacentpoints on said simplified reference surface RS. The centroids, nodes orthe considered voxels of the chosen surface of reference are called“Reference Points” hereafter.

Three-dimensional surface segmentation techniques, and techniques todiscretise the surface, are well known and so will not be described indetail here. Further information on segmentation can be found in the“Handbook of Medical Imaging, Processing and Analysis”, editor-in-chiefIsaac N. Bankman, Academic Press, chapter 5 “Overview and Fundamentalsof Medical Image Segmentation” by Jadwiga Rogowska.

S3 for constructing a distance transform map. In step S3, surfaces,called “Distance Transform Surfaces”, denoted by DT, are calculated.These surfaces are distance transforms of the reference surface RS. Thereference points of the reference surface are propagated as well astheir labels, either outwardly by a dilation operation, or inwardly by acontraction operation, yielding one or several distance transformsurfaces DT, each within a given distance from the reference surface RS.As illustrated by FIG. 3A, to a reference surface RS, correspond theoutward distance transform surfaces DT11 and DT12, and the inwarddistance transform surfaces DT21 and DT22. To each reference point (A,B, etc.) of the reference surface RS corresponds a unique image point oneach distance transform surface DT. Moreover, to each point on eachdistance transform surface DT, a label of its corresponding referencecpoint on the reference surface is assigned. As illustrated by FIG. 3B,to the reference point A of the reference surface RS, correspond imagepoints A′, A″, A″′ on the distance transform surfaces DT11, DT12, DT13.Since these image points A′, A″, A″′ are located, on the normal N_(A) tothe reference surface RS at the reference point A, and on the distancetransform surfaces DT11, DT12, DT13, it results that these image pointsA′, A″, A″′ are located at given predetermined distances from saidreference surface RS, and that these image points A′, A″, A″′ arerespectively the unique correspondent of said reference point A on saiddistance transform surfaces DT11, DT12, DT13, etc.

In the same way, the normal NB at reference point B, shows the imagepoints B′, B″ on the distance transform surfaces DT11, DT12, with thesame properties.

In the present invention, clinical data are to be displayed associatedwith reference points, A or B, etc. These clinical data are evaluated atthe location of the image points, A′ or B′; A″ or B″, etc, located alongthe normal N_(A) or N_(B), etc, to the reference surface RS, at theintersection with the different distance transform surfaces DT, asdescribed above and illustrated by FIG. 3A. Hence, the present inventiondeparts from the Zuiderveld et al. approach, because the image pointsare not only located along a surface normal, but also on the differentdistance transform surfaces, at different given distances from thereference surface RS that are predetermined by the construction of saiddistance transform surfaces.

According to preferred embodiments of the invention, image points aredetermined along the surface normals corresponding to every referencepoints, at the intersection with the distance transform surfaces. So, animage point of a distance transform surface corresponds univocally to areference point of the reference surface. The image points closest tothe surface of interest are first identified, then the image pointsfurther and further away on the different distance transform surfacesare identified, as far as possible from the reference surface.Preferably the image points are selected both along the surface normaland along the reverse surface normal. The different identified imagepoints corresponding to the reference point of the reference surface RSmodelling the clinical surface of interest, located on said distancetransform surfaces, will constitute a map of points, called “datadistance map”, which is formed of image points surrounding the referencesurface outwardly and inwardly.

The main advantages of the present invention stem from the creation ofsaid “distance map”. The properties of the map are as follows: The mapensures the “uniqueness” of the image points with respect to thecorresponding reference points, due to the fact that, in each distancetransform surface, a single image data point corresponds to onereference point of the reference surface. The map ensures the “orderconservation”, due to the fact that the relative positions of a firstand a second image data points on any given distance transform surface,are the same as the relative positions of the corresponding first andsecond reference points on the reference surface.

However, further tests may be performed to better select the points ofthe map, in order to still improve the above-described imagingtechnique. Tests are proposed bellow for selecting the image points thatwill preferably be taken into account when making the evaluation ofclinical data associated with a reference point. Among the proposedtests:

A magnification test: A first test called magnification test,illustrated by FIG. 3C, may be performed in order to ensure that thedistances (in directions parallel to the surface of interest) betweenimage data points that are taken into account when evaluating clinicaldata associated with reference points of the reference surface are keptwithin user-defined ratios. For instance, regarding the points A′, B′,which correspond to A, B, the magnification test has means for computingthe value of A′B′/AB and for estimating whether said value is within apredetermined range of values, and means to eliminate the points thatfail the test.

A distance test: A second test, called distance test, illustrated byFIG. 3B, may be performed in order to ensure that *each image datapoints, which is taken into account when evaluating clinical data, isassociated with the closest reference point of the reference surface.This distance test is only needed when distance transform surfaces DTare created without a point labelling technique, such as the pointlabelling technique described above. Generally, according to theinvention, it is sought to select points of the normals to the referencesurface, which are on distance transform surfaces positioned as far aspossible from the reference surface. However, the farthest found imagepoint, which corresponds to a given reference point of the referencesurface, must not be located nearer to another reference point than toits own corresponding reference point. For instance, the image point A″′on DT13, which corresponds to the reference point A, would be nearer tothe reference point B than to its own corresponding reference point A.The distance test ensures that such an image point A″′ cannot not becoupled with B when constructing the map. Hence, A″′ is discarded. Thistest gives the ultimate image point that is selected on a given normal.

It results from the application of these tests, that a number of imagepoints of the distance map are deemed necessary to be rejected in orderto improve the imaging technique. Hence, said “distance map”, may nothave a uniform thickness or may not have the same thickness each side ofthe surface of reference.

The first three properties are inherent to the construction of thedistance map, since in said construction, by dilation or contraction,each point of the constructed distance transform surfaces corresponds toa single original reference point, which ensure the uniqueness of theimage points, the conservation of relative position of the image pointsand the conservation of shape of features formed of image points. Thanksto the use of the distance map, the present invention ensures that asingle data point cannot give rise to data visualised at two differentplaces on the anatomical surface of interest. Hence, the inventionreduces ambiguity in the integrated representation of the anatomicalsurface of interest and the associated clinical data. Thanks to the useof the distance map, the present invention ensures that differentclinical data items that are visualised in association with theanatomical surface of interest are in relative positions, which reflectthe true relative positions of these data points in the patient's body.By rejecting image data points which fail the proposed magnificationtest, and/or which fail the distance test, the preferred embodiments ofthe present invention ensure that when the clinical data are visualised,the apparent size of any feature (e.g. a region of increased thickness)is not unduly exaggerated or minimised. According to the invention, theuse of the map of data points permits to avoid artefacts that render thevisualised image ambiguous.

S4 for evaluating the clinical data linked to the image points of the“distance map”. According to the present invention, the image datarelating to the surface of interest are to be displayed in an integratedfashion with associated clinical data. Thus, it is necessary todetermine which clinical data is to be visualised in association withthe respective reference points A, B, etc. of the reference surface RS,approximating the surface of interest.

The clinical data for display are determined indifferently before orafter performing an operation of surface rendering for providing saidspecific reference surface RS (reference polyhedron, simplified surfaceor any other kind of smoothed or discretised surface representative ofthe surface of interest), to be chosen as a support for displaying saiddata in an integrated manner, and to be constructed by using one of theabove-described techniques.

In step S4 illustrated by FIG. 4A, the clinical data to be visualised inan integrated fashion with the reference surface are evaluated at thelocation of the selected image points of the “distance map” defined instep S3. This evaluation can calculate a value for various differentclinical data, for example, the minimum intensity projection, themaximum intensity projection, the mean intensity projection, or the sumof intensities along the normal. The “minimum intensity projection”value for a given reference point is the lowest intensity value amongthe image points that are located along the normal at the referencepoint and that are within the “distance map” defined in step S3. The“maximum intensity projection” and the “mean intensity projection” and“sum of intensities” are self-explanatory.

The clinical data evaluated at the location of the image points of the“Distance Map”, further form an “Associated Data Distance Map” thatwraps the reference surface outwardly and/or inwardly.

S5 for clinical data coding. In step S5, once the clinical data havebeen evaluated for the various image points of the “distance map”, thecalculated values are encoded, for example into colour code values, tobe visualised in an integrated fashion with the image data of thereference surface RS representing the clinical surface of interest. Theclinical data can be encoded in a variety of ways, for example, usingcode values which produce different patterning, colour or texture on adisplay of the surface of interest. If colour coding is used, this canfollow various approaches, for example a Red-Green-Blue (RGB) approach,or a hue-saturation-value (HSV) approach. The present invention isapplicable regardless of the manner in which the clinical data areencoded and visualised in association with the reference surface.

S6 for combining data. In step S6, then, the encoded clinical data ofthe associated data distance map and the rendered surface data of thesurface of reference representing the anatomical surface of interest arecombined, so as to be output. So, the encoded clinical data evaluated atimage points on a given normal are combined with the image data at thelocation of the corresponding reference point on the reference surface.

Image Data Output for Visualisation: In general, the combined outputdata are displayed on a display device such as the monitor 40 of themedical viewing system of FIG. 2. The evaluated clinical data can betime-varying data. For example, the rate of perfusion of a contrastproduct into the myocardium is of clinical interest. This can berepresented by gathering image data over time, as the contrast productenters the myocardium, evaluating the maximum/minimum intensityprojection along the normals at the different reference points of areference surface approximating the myocardium at different moments, andcolour encoding the calculated values. The user will obtain arepresentation of the myocardium with a changing pattern of coloursshowing the perfusion of the contrast product.

In a preferred embodiment, further described with reference to FIG. 4B,the method comprises sub-steps of the above-cited Step 2. In sub-stepS21, the reference surface RS is constructed. In sub-step S22, thereference points are labelled. In a sub-step S23, the reference pointsare validated. A predetermined distance controls the resolution of thedata to be visualised in association with the anatomical surface ofinterest. A limitation value may be set taking into account the clinicaldata of interest and anatomical considerations (if the distance is toolarge, data would be unduly considered, whereas they relates to organsor anatomical features other than those of interest). Then, eachreference point of the reference surface is processed in turn and thenormal to the reference surface is calculated at each reference point.

Then step S3, is performed as previously described. The “DistanceTransform Surfaces” DT are constructed. The tests of selection of theimage points forming the map are performed. At the end of the testingprocedure described above, the distance map of valid image data pointshas been constructed in correspondence to the reference surfacemodelling the anatomical surface of interest.

In a preferred embodiment, described with reference to FIG. 4B, themethod comprises sub-steps of the above-cited Step S4. In sub-step S41,a list of the valid image points is issued. In sub-step S42, clinicaldata are evaluated. The original medical image data are sampled at eachof the valid data points. In general it is necessary to performinterpolation between voxels in the original image data, because thelocations of image points does not necessarily coincide with thelocations of the voxels in the original medical image data. In sub-stepS43 the clinical data are positioned. The set of sampled data representsthe valid data that can be analysed in association with respectivepoints of the reference 3D surface, in order to evaluate clinical datathat are to be visualised in an integrated fashion with the anatomicalsurface of interest. In a sub-step S44, the “Associated Data distanceMap” is formed. The Associated Data distance Map represents areformatting of the original medical image data, for instance areformatted volume of image data.

In the above description, it is assumed that the clinical data and theassociated anatomical surface of interest will be visualised in anintegrated fashion in 3-D form. However, optionally, the reference 3-Dsurface can be flattened, the object interface can be estimated byrepresenting it in the reformatted volume as a regular function (forexample, B-spline) in the mathematically simple form: w=f(u,v), where wis the signed normal distance to the reference 3-D surface and (u,v) arethe coordinates in the reference 3-D surface. Standard best-fitprocedures can be used when working with this simplified representation.Alternatively, or additionally, the image intensities projected onto theflattened reference 3-D surface representation form a 2-D image that canprovide useful information in its own right. For example, the 2-D imagecan be processed using known 2-D handling techniques in order to analysevessel width or vessel stenosis, or in order to determine the vesselcentreline.

In the above-described preferred embodiment, 3-D medical image data wereobtained via computed tomography apparatus. It is to be understood thatthe present invention is applicable regardless of the medical imagingtechnology that is used to generate the initial data. For example, whenseeking to visualise the heart, magnetic resonance (MR) coronaryangiography may be used to generate 3D medical image data in anon-invasive manner. See, for example, “Non-invasive CoronaryAngiography by Contrast-Enhanced Electron Beam Computed Tomography” byAchenbach et al, in Clinical Cardiology, 21, 323-330, 1998. TheAchenbach et al article includes useful information regarding optionaldata processing steps that can be applied to the medical image data, forexample, segmentation to enable a representation of certain anatomicalfeatures in isolation from others, details of shading techniques used toproduce a displayed image, etc. These steps can be applied in the methodof the present invention.

The present invention is applicable regardless of the way in which theanatomical surface of interest is modelled, whether via use of areference polyhedron, use of a reference simplex mesh, or in some otherway. Preferably, the anatomical surface of interest is merely identifiedin the image data via a segmentation step followed by a smoothing step,which provide the reference surface RS, and there is no specificmodelling of the identified surface.

Various modifications can be made to the order in which processing stepsare performed in the above-described specific embodiment. Theabove-described processing steps applied to medical image data canadvantageously be combined with various other knownprocessing/visualisation techniques. For example, it is known whenmodelling a surface by a reference polyhedron or mesh for image analysisand visualisation, to set the facet size adaptively, typically so thatthe facet sizes are not too large (which would give poor spatialresolution). It can be advantageous to apply this adaptive setting offacet size in the present invention for the same reason, as well as toavoid the case where each facet has few or no corresponding voxels.

The drawings and their description hereinbefore illustrate rather thanlimit the invention. It will be evident that there are numerousalternatives that fall within the scope of the appended claims. In thisrespect the following closing remarks are made.

Moreover, although the present invention has been described in terms ofgenerating image data for display, the present invention is intended tocover substantially any form of visualisation of the image dataincluding, but not limited to, display on a display device, andprinting. Any reference sign in a claim should not be construed aslimiting the claim.

1. A medical viewing system comprising, data acquisition means foracquiring image data in an image of an object surface; processing meansfor integrating clinical data with the image data, the processing meansfor integrating comprising processing means for processing the imagedata to identify a reference surface approximating the object surfaceand reference points on said reference surface; means for constructing adistance map comprising one or more distance transformed surface(s),from the reference surface, formed of image points that correspondunivocally to reference points of the reference surface; means forestimating, at the location of the image points of the map, clinicaldata, and means for combining the clinical data and the image data atthe location of the reference points, to integrate the clinical data inthe image data; and said medical viewing system further comprising imagevisualization means for visualizing the object images or the processedimages.
 2. A medical viewing system according to claim 1, furthercomprising processing means to encode the clinical data, so as tovisually differentiate them from other data.
 3. A medical viewing systemaccording to claim 1, wherein the image points of the distancetransformed surface(s) are each located at the intersection of a normalto the reference surface at a reference point and said distancetransformed surface(s), whereby the transformation ensures theuniqueness of the corresponding points of the distance transformedsurface(s) with respect to the reference points of the reference surfaceand the conservation of relative positions of the points of anatomicalfeatures.
 4. A medical viewing system according to claim 3, furthercomprising testing means for testing the image points of the distancemap, among which a magnification test for estimating whether, indirections parallel to the reference surface, the ratio of the distancebetween two image points of a distance transform surface, to thedistance between the corresponding points of the reference surface, iskept within a predetermined range; and selection means for discardingpoints of said distance transform surface that fail the magnificationtest.
 5. A medical viewing system according to claim 4, wherein thetesting means for testing the image points of the distance map performsa distance test for estimating whether, in directions orthogonal to thereference surface, an image point on the normal to the referencesurface, located on a distance transform surface, is closest to thecorresponding reference point on the reference surface or closer toanother reference point on said reference surface; and furthercomprising selection means for discarding points of said surface normalthat fail the distance test.
 6. A medical viewing system according toclaim 5, further comprising, processing means for computing clinicaldata at the location of the image points of the distance map, so as toform an associated data distance map; means for combining said computedclinical data of the associated distance map with the image data of thecorresponding reference points of the reference surface, so as theclinical data of image points on a given normal to the reference surfaceis combined with the image data of the reference point corresponding tosaid given normal; and means for displaying respectively the combineddata on the reference surface at the location of the correspondingreference points.
 7. A medical viewing system according to claim 6,further comprising processing means for: segmenting the image datawhereby to identify the object surface of an original image;approximating said segmented object surface data for determining thereference surface, which represents an approximated surface of saidobject surface devoid of folded portions; determining reference pointson the reference surface; and calculating the normals to the referencesurface at the reference points.
 8. A medical viewing system accordingto claim 7, further comprising processing means for constructingdistance transform surfaces from the reference surface, by dilation orcontraction of the reference surface, so as to form a map of imagepoints that is the distance map, and that wraps the reference surfaceoutwardly or inwardly, each image points of the distance transformsurfaces corresponding univocally to a reference point and being locatedat the intersection of a normal to the reference surface and a distancetransform surface.
 9. A medical viewing system according to claim 8,further comprising processing means for constructing the referencesurface as a smoothed simplified surface from the segmented surface, andidentifying the reference points as points of said smoothed simplifiedsurface.
 10. A medical viewing system according to claim 9, furthercomprising processing means for constructing the reference surface as adiscretised surface from the smoothed simplified surface, said referencesurface showing a plurality of facets or patches.
 11. A medical viewingsystem according to claim 10, further comprising processing means forgenerating a flattened 2-D representation of said reference surface. 12.An image processing method to cause the data processing means of themedical viewing system of claim 1 to perform the steps of acquiring andprocessing image data in an object image of an object, for integratingclinical data with the image data, wherein processing comprises:processing the image data, whereby to identify a reference surfaceapproximating the object surface and reference points on said referencesurface; constructing a distance map comprising one or more distancetransformed surface(s), from the reference surface, formed of imagepoints that correspond univocally to reference points of the referencesurface; estimating, at the location of the image points of the map,clinical data, combining the clinical data and the image data at thelocation of the reference points, so that to integrate the clinical datain the image data; and visualising the object images or the processedimages.
 13. A medical examination apparatus according to claim 1,further comprising acquisition means for acquiring medical image dataand imaging means for displaying the medical images.
 14. A computerprogram product having a set of instructions, when in use on ageneral-purpose computer, to cause the computer to perform the steps ofthe method according to claim 12.